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Abstract. Let c*; be a point in the upper half plane, and let F be a discrete, finite covolume 
subgroup of PSL2(R). We conjecture an explicit formula for the pair correlation of the angles 
between geodesic rays of the lattice Fa;, intersected with increasingly large balls centered at cj. We 
prove this conjecture for F — PSL2(Z) and uj an elliptic point. 
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1. Introduction 

The statistics of spacings measure the fine structure of sequences of real numbers, going beyond 
the classical Weyl uniform distribution. Originating in work of physicists on random matrices 
[20, 8], spacing statistics are conveniently expressed as the convergence of certain measures, called 
level correlations, and respectively level spacing measures. In the past decades these notions have 
received significant attention in many areas of mathematical physics, analysis, probability, and 
number theory. For most sequences of interest it is usually very challenging to prove the existence 
and describe the limiting spacing measures, such as the gap distribution or pair correlation, even 
when their existence is experimentally predicted. 

One class of interesting sequences studied in recent years arises from the angular distribution of 
lattice points. In the Euclidean scenery one such question is: for a given point a E M^, describe 
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the statistics of the increasing sequence of finite sets 

: m G \ {-a}, |m + a| < i? I C 5\ with oo, 
|m + a| J 

representing the directions of points in the affine lattice a + with observer located at the origin. 
When a ^ the gap distribution of this sequence was proved by Marklof and Strombergsson 
[15] to coincide with the gap distribution of the sequence {y/n mod 1), previously computed by 
Elkies and McMullen [9]. In this situation we are not aware of any result concerning the pair 
correlation for an individual a. When a S the limiting gap distribution was computed in [2]. A 
major difference between the two situations is that the gap distribution density, always supported 
in [0,oo), vanishes on [O, when q G and is equal to on [0, ^] when a ^ Q^. This kind 
of repulsion can be used to explicitly compute the pair correlation when a E Z^. When the disk is 
replaced by the square [—R, R]"^ and one only considers primitive lattice points, the pair correlation 
was computed in [4]. 

In the hyperbolic situation, the lattice Z^ is replaced with a lattice (discrete subgroup of finite 
covolume) F in PSL2(K). We consider the angles between geodesic rays — )• 70;) in the upper half 
plane H, connecting a fixed point a; € H with the (finitely many) points 70; in its F-orbit, lying in 
increasingly large hyperbolic balls. These angles are well-known to be uniformly distributed (see, 
e.g., [16]) and their uniform distribution in angular sectors can be made effective [1, 6, 7, 11, 12, 
13, 17, 18]. 

A first step in the study of the pair correlation of directions of hyperbolic lattice points was 
completed in [5], where we treated the case T = PSL2(Z) and lo = i, establishing a formula for 
the pair correlation density §2(0 that involves two terms. The first term is a series over the set 
of matrices M with nonnegative entries of an explicit function of depending only on the Hilbert- 
Schmidt norm of M, while the second term is a finite sum involving volumes of bodies defined 
in terms of the triangle transformation introduced in [3]. In this paper we extend the approach 
introduced in [5], conjecturing an explicit formula for the pair correlation density 52(0 arbitrary 
r and u, and we prove the formula for the modular group and uj an elliptic point. Remarkably, we 
find that g2iO equals the diagonal value at u of an explicit automorphic kernel. 

To state the results we introduce some notation and definitions. Let oj = u + iv £M and let T 
be a discrete, finite covolume subgroup of PSL2(M). For 7 G F, define ||7|| := vy^2 cosh d{ijj, 70;), 
where d{zi,Z2) denotes the hyperbolic distance between two points zi,Z2 € H. Let Bq^ be the 
number of matrices 7 G F in the ball ||7|| ^ Q, so that asymptotically Bq^ ~ ^Q"^- 
interested in the pair correlation density function 

92(0 =^^, where i?2(e) = and 

T^QiO = #{(7,7') e r2 :7' / 7, II7II < Q, ||7'|| ^ Q , ^ ^{Oy - 6,) ^ -^|. 

The approach and computations from this paper provide evidence toward the following: 

Conjecture 1. Let T be a discrete subgroup o/PSL2(M) with fundamental domain of finite area 
Vr- The pair correlation measure -^2(0 exists on [0,oo), and is given by a function expressed 
as a series of three dimensional volumes. Its density 92 is given by the formula 

3{Pi = ^Efd^m, (1-1) 
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where i{M) = d{uj, Muj) and f^{i) is the continuous function defined for £ ^ and > by 



In ( (cosh^+sinh£)(i+g^) \ ,/2sinh(|)^e^smh^, (l-^) 

In (cosh£ + sinh-f) = l if sinh£ ^ ^. 

Since the series above is absolutely convergent, by I'Hospital we also deduce the conjectural 
formula: 

M6r/{A/)>0 

For the elliptic points for PSL2(Z) we prove the conjecture, using extra symmetries of the hy- 
perbolic lattices centered at i and p = e^"^^^ . 

Theorem 1. Conjecture 1 and formula (1.3) hold for V = PSL2(Z) and for uj one of the elliptic 
points i or p (with Vy = tt/SJ. 

The conjecture has also been verified numerically for a few congruence subgroups ro(A^) and a 
few points u. In Figure 1, we compare the pair correlation function given by (1.1) with the actual 
pair correlation function computed by counting the pairs in the definition, for F = PSL2(Z) and 
for a few choices of oo. To count the pairs (7,7') in the definition of TZq{^), we first reduce to a 
half ball |7w|, |7'ti;| as explained in Section 3. 

When F = PSL2(Z) and u is one of the elliptic points i or respectively p, each angle in the 
definition of TZq{£^) is counted a number of times equal to the order of the stabilizer of lo in F, 
namely 2 or respectively 3 times. Therefore it is more natural to consider the pair correlation 
measure TZ°q defined as TZq with the condition 7 7^ 7' replaced by ^uj 7^ 7'a;. Denoting by §2 the 
corresponding pair correlation functions, we have (72HO = g2{^u)i) where e^j is the cardinality of the 
stabilizer of w, so that §2 = 52 if w is not an elliptic point. For to = i, the function §2^ is identical 
with the pair correlation function found in [5], but the formula here is entirely explicit for all ^. 

Formula (1.1) relates the pair correlation of hyperbolic lattice angles with the length spectrum 
of the lattice. For example, the spikes in the graphs in Figure 1 occur at values of ^ related in a 
straightforward way to the length spectrum. Assuming that lattices centered at different points 
in a half^ fundamental domain for PSL2(Z) have different length spectra, it would follow that the 
distribution of lattice angles determines the point a; in a half fundamental domain. 

A common feature of pair correlation density functions, encountered also for the pair correlation 
of Farey fractions [4] , is that they tend to one at infinity. We expect that the same is true for the 
function in Conjecture 1, namely that if F is a Fuchsian group of the first kind with fundamental 
domain of finite hyperbolic area Vp, then 

l^^Y.fddi^^l^))=l, (1-4) 
A/er 

where the function f^{i) is defined in Conjecture 1. The standard way of proving an asymptotic 
formula such as (1.4) is by using the spectral decomposition of the automorphic kernel K^{z,w) = 
^^gP /cg(n(7z, i(;)), where k^{u{z,w)) = f^{d{z,w)), and we attempt to spectrally decompose the 
kernel in Appendix B. However, we encounter technical problems due to the fact that the kernel 



-'^It is shown in Section 3 that the pair correlation functions for the lattices centered at lu and —lj are equal. 
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k^{u) is not smooth at two points, and as a consequence we can only prove that its Selberg-Harish- 
Chandra transform h^{t) satisfies h^{t) ^ (1 + for real t. This is an epsilon short of the decay 
(1 + for some e > 0, required for absolute convergence in the spectral decomposition of K^, 

and therefore we leave the proof of (1.4) for later work. 




^ OS 1 TS 2 275 3 ^ 03 1 Ts 2 275 3 

Figure 1. The pair correlation functions for F = PSL2(Z) and Re(ci;) = u, 
\uj\ = k computed using (1.1) (smooth line) and by counting pairs in the definition 
(jagged line) with Q = 1000 in the first two plots, Q = 2000 in the third, and 
Q = 5000 in the fourth. 



For F = PSL2(Z) and co = i, the pair correlation function g^{C) is the same as that of angles 
made by reciprocal geodesies on the modular surface, namely the closed geodesies passing through 
the projection of i on the modular surface. Reciprocal geodesies were first studied by Fricke and 
Klein [10], and more recently by Sarnak [19]. In Appendix A we similarly describe the arithmetic 
and geometry of closed geodesies passing through the projection of p on the modular surface. While 
reciprocal geodesies always consist of two loops, one tracing the other in the opposite direction, we 
show that a geodesic on the modular surface passing through the image of p consists of one, two, or 
four closed loops. The precise situation depends on the arithmetic properties of the discriminant 
attached to the geodesic. 

We now sketch the main steps in the proof of Theorem 1, while also describing the organization 
of the paper. In the remaining of the introduction and throughout the paper we take F = PSL2(Z), 
and keep u mostly arbitrary. For technical reasons, we assume that Re(a;) and \uj\'^ are rational. 
An important role is played by the set © of matrices with nonnegative entries, distinct from the 
identity. 
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Step 1. As in [5], our approach is based on computing the pair correlation of the quantities 
^'(7) = u + utan (-^j by first approximating them with $(7) = Re(7a;). The reason for preferring 
the function $(7) is explained by Lemma 1, where we show that $(7) — $(7M) = Ha./(c, d) with 
a function Sm depending only on the lower row (c, d) of 7. In Section 3 we reduce to angles in 
the hyperbolic half balls for which \"foj\ < and we show in Section 4 that the sets {\I'(7)} and 
{<1>(7)} have the same pair correlation. 

Step 2. To compute the pair correlation of {<l>(7)}, we estimate the number 7^q(^) of pairs 
(7,7') G with |7a;|, |7'w| < k, \\^\\, \\^'\\ ^ Q, and ^ <l>(7) - $(7') < ^ as follows: 

KiO = \ E (1-5) 

where Mm,q{C) is the cardinality of the set 



2 

Afer\{/} 



SM,Q{i) := <! 7 G r : |$(7M) - ci>(7)| ^ ^, |7a;|, |7Ma;| < k, hH, ^Af || ^Q). (1.6) 



Replacing h in terms of a,c,d, for M G (3 we show that N'm^q{£,) is asymptotic as Q — )• 00 to the 
cardinality N'm,q{0 of t^ie set Sm,q{0 of integer triples {a,c,d) such that 



\a\ ^ kc ^ Q, \d\ ^ Q, ad = 1 (mod c), \Em{c, d)\ ^ 

_max{c2/c2 + ^ 2c(iu, C^Xm + d^n.f + 2cdZM} ^ a^+£f_2acu ' 



(1.7) 



with Q = — and Xm,Ym, Zm defined in (2.1). This approximation holds for fixed M G (3 

v^k-\u\ 

with an explicit error term (Lemma 9 (iii)), but in order to control the error when when summing the 
series (1.5) we need to replace A/'m,q(0 by the cardinality J^m giO ^^e subset S^f qiO C 5m,q(0 
consisting of triples (a, c, d) as above with d > 0. 

Step 3. Using estimates for number of points in hyperbolic regions based on bounds on Kloost- 
erman sums, we show in Lemmas 10 and 11 that for M G 6, Mm,q{C) ~ ^) VoK^'a/,^)) and a 
similar estimate holds also when summing J^^f q{0 o^^r M G ©. The region Sm,(, consists of triples 

5iep ^. Using extra symmetries of the hyperbolic lattice in the case uj = i (Section 8) and u = p 
(Section 10), we show that the summation range in (1.5) can be reduced to a subset of ©. Moreover, 
using repulsion arguments involving the Farey tessellation, we can define finite subsets -F(^) G F 
such that for M G the quantities A/a/,q(^) can be expressed only in terms of AA^^ q(^) for 

appropriate G S. Therefore we place ourselves in the situations analysed in Steps 2 and 3, 
obtaining 

-wk^ Vo1(5m,^). (1.9) 
^ a/gf 

Step 5. The resulting volumes are expressed in closed form as integrals in Section 9 for uj 
arbitrary, and passing to the pair correlation function R2 of the angles {^-y} we obtain the formula 
in Conjecture 1, and finish the proof of Theorem 1. 

The difficulty in proving Conjecture 1 for F = PSL2(Z) and general u resides in the estimates 
of Steps 2 and 3, where we use positivity of some of the entries of the matrices involved to obtain 
good control on the error in lattice point counting. It would be interesting to find a proof that 
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involves the spectral theory of the hyperbolic Laplacian. For the problem of equidistribution of the 
geodesic angles, such an approach is provided for example by the estimate for exponential sums in 
[11, Theorem 4] (restated in a more transparent way in [6, Theorem 1.1]). For the pair correlation 
problem we would need a suitable extension of these results, instead of the lattice point counting 
arguments used in this paper. Such an approach could potentially generalize to Fuchsian groups of 
the first kind. 



2. Preliminary computations 
To each lo = u + iv M with \uj\ = k and 7 = ( " ^) S SL2(M), we associate the quantities 
X = \au + b\'^, Y = \ao + d\'^, Z = Y Rei'yuj) = ac\uj\'^ + bd + u{ad + be) , 

T =\\-f\\'^ := X + \uj\^Y -2uZ. ^'^'^^ 



O 'T' /rTi2 A 2 

Setting A = 2v and = ~ a~ — > have 

XY - Z"^ = v^, cosh d{u}, 70;) = ^, (2.2) 

Z-uY ^ AY-T {e.A IZ-uY , , 

smt^^ = 2v , , cost/^ = =, tan — = —— . 2.3 

^ - A2 ' ^ VT2 - A2 ' \2 ) VY-ET 

The point 70; is completely determined by the "coordinates" (X, y, Z) = (X^,Ky,Z^), as its hy- 
perbolic polar coordinates are so determined. 

The x-intercept 'J/ (7) of the oriented geodesic cj 70; is given by 

\ Z^ — uct 



\ I J — ex 

Since ct = e~^^'^'"''^^ — )■ 0, a better behaved quantity which approximates ^'(7) well is 

^(7) := =Re(7^^). 

For M = [cd) ^ let £(M) = d{uj,Mu), and let the angle Om be defined as for 7. For 
c, d G M, let ccj + d = re*^, and define 

- (r rl-)— ^ ^'"^^^^^ ~ (^d^ 

-M[c, a) . ^2 coth£(M) + cos(0M - 20) ' ^ ' 

Lemma 1. For M £ SL2(M) and j = (^ ^) G SL2(M), we have 

$(7)-$(7M) = SAf(c,d). 

Proof. We compute Re(7a; — 'jMu) using the KAK decomposition for M and the NAK decom- 
position for 7, both centered at uj. Denote 

, , /yi/2 \ , / N A /cos6l -sin6'\ 

and let a = n{u)a{v), so that ai = co and a maps the vertical through i onto the vertical through 
uj. Let z := a~-'^7ti;, so 

= az = u + vz, z = X + iy. (2.5) 

We have 

a~^^a = n{x)a{y)k{0), with euj + d = re*^, r = y^^^^. 
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Let O'j^.^ be the angle between i — )• ioo and i — )• a~^Mai. Since a maps i — )• a~^Mai onto to — )• Muj^ 
we have 

e'M = TT- Om, d{i, a~^Mai) = 1{M). 

Consequently 

a-^Ma = k{e'M/2)a{e'^^'^)k{e") 



for some 9", and taking u = 9 + \9'm we obtain 



a-^-fMuj = n{x)a{y)k{9 + 9'^ /2)a{e^^^'^^)i = x + y 



ie^(^) cos u — sin 



Re{-yMuj) = Re(7a;) + ^ Re 



sin u + cos u 

The equation above and (2.5) imply 

' ie^^^^ cos u — sin u ' 
sin u + cos i^^ 

The real part in the last expression equals the second fraction in (2.4), and the claim follows. □ 
A direction calculation yields 

„ . _ cdjk^YM - Xm) + c'ik^ZM - uXm) + cP{uYm - Zm) 
^ {c^k^ + d^ + 2cdu){c^XM + d^YM + 2cdZM) 

3. Reduction to angles with |7a;| < |a;| 

We next show that the pair correlation function is determined only by the angles 9^ with \-yuj\ < 
\uj\ = k, justifying the assumption made in the introduction. When k ^ 1, our proof is conditional 
upon the formula in Conjecture 1. 

We denote by the set of 7 G T with ||7|| < Q, of cardinality Bq^, and let £Hq, respectively 
£Hq denote the subsets for which |7a;| < k, respectively \juj\ > k, oi cardinalities Bq, respectively 
Bq. It is shown in Lemma 8 that Bq ~ Bq ~ -^Q^. Let 



^qHO = #{(7,7') G {^'q? : tV 7, ^ ^{9y - 9. 
,tot... _ ^1^(0 tot ^ d^' 



7; ^ rjtot 



and define similarly i?^, 5^. Since Bq B> ^ i-^g*, we have that Rl°^ = i(i?2 + ^^), 52°* = 

Let s = (? ) • Since we will compare the hyperbolic lattices centered at the points uj and sco, in 
the following two paragraphs only we attach subscripts to all notation to denote this dependence, 

6g 1 1 7 1 1 tiJ ) 

(Bq)^ etc. 

The map 7 1— )• 575 induces the mapping 70; 1— )• 570;, taking the part of the lattice Tu with 
\juj\ > \u}\, bijectively and conformally onto the part of the lattice Tsuj with |7sa;| < |sa;|. Note 
that 

k (^S7s)sa; (^7)1^5 k {Yg.yg'jgi^ (Yy'ji^, k (yZg^g^g^ {^Z-y)ijj^ 

k ||s7s||saj — ||7||(^5 (^S7s)saj — (^7)0)5 

yielding 

{Bq)ui = {BQjk2)gui, (TiQ)ui = {T^Q/k^)sui, R2(.0u; = R2{0sui, 92iOu> = 92{0sui- 

We conclude that 52(0^°* = 5 (52(60. + 52(0 so;) • 
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Assuming now that g2(0t<^ is given by the series in Conjecture 1, we observe that the apphcation 
M —7- sMs rearranges the terms of the series for g2{C)w into the terms of the series for 52(0sa; 
because the summands only depend on d{uj,Muj). Therefore g2{C)oj = 92{^)sw, and hence we have 

5r(6 = 92(o = 92>(e) 

(dropping the subscripts uj since the basepoint of the lattice is fixed). 

When = 1, one can see directly that g2{i) = 92 iO^ because of an extra symmetry of the 

?i)andS = |i. 



hyperbolic lattice. Keeping w arbitrary, let 7 = 77777 for 77 = ( 5 n) ^^^"^ ^ = ifj- Since 



1"^ J^2 ' y-'^-yJi^ ^2 ' y^lJi^ f^2 

we have \^u)\ < k \^u}\ > k and 

The angles {0^)i_j and {6^)[:j are related as in the following lemma, which shows directly that 92(0 = 
92 when \uj\ = 1 (that is a; = u). 

Lemma 2. Let /3 G (0,vr) be the polar angle of uj = ke^^ . We have the relation: 

(^7)- + (e^h = 2/3, (3.1) 

namely the angle between the circle \z\ = k and lo — ?• 7^ is the same as the angle between the circle 
\z\ = k~^ and oj — t- 7S;. 

Proof. When ui = ki the claim is immediate from (2.3). In general, let a= {""l^^^^) G SL2(K) for 

^ ~ \/ fep'' ^ ~ \/ ^JP' ^S'-^ ^' Then a fixes the circle \z\ = k and takes ki to w, while 5 fixes the 
circle |2;| = k~^ and takes A;~^i to uj, therefore 

Z{uj — )• 70;, cj — )• fc) = Z(A;i — )• a~"^7aA;i, ki — )• fc). 

Likewise one has 

Z(a; — 7- 70;, —7- k~^) = Z{k~^i — t- oT^^ak^^i, k~^i — ?• A;^"^), 

and the last angles in both equalities are equal by the case uj = ki already proved. 

Alternatively, equality (3.1) can be checked by direct computation, using the formula for tan(a + 
b). □ 

From the lemma, combined with 92 (0<^ = 92(0*'^ '^^ also deduce that the pair correlation 
functions for the hyperbolic lattices centered aX uj = u + iv and suj = —u + iv are equal. This shows 
that we can restrict ourselves, whenever convenient, to points uj in the half fundamental domain 
for r given by 

\u}\^l, Re(a;)^0, - 1| ^ 1. (3.2) 

4. The coincidence of the pair correlations of <I> and ^' 

Since the pair correlation of the lattices centered at uj and 790; is the same for 70 G T = PSL2(Z), 
in this section we assume without loss of generality that uj lies in a specific fundamental domain for 
the action of T on the upper half plane. Namely, we assume that ^ Re(a;) ^ 1 and jf^ — ^| > ^ 
that is 

u2 



2' 



^ n ^ 1, r > u. (4.1) 
We also need to assume that n, k"^ G Q, which is needed in the proof of Lemma 3. 
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Next we show that ^ have the same pair correlation. As in Section 3, let D\q be the set of 
7 G r with I'yul < k and ||7|| ^ Q. Consider 

:= # {(7,7 ) e {9\Qf : 7 / 7, ^ <I>(7) - $(7') < ^} 

and the likewise defined TZq ) . 

Let 7=(crf)£r, II7II ^ Q and X, Y, Z, T be the quantities defined in the beginning of Section 
2. By the results of Section 3 we can restrict to those 7 E F such that |7aj| < k, that is X < k'^Y. 
In this case we have 

^ = + ^ - 2u^ < 2k^ + 2A;|n| < 1, 
and employing \Z — uY\ ^ T, a consequence of the first formula (2.3), we have 

In — — I 1 1 1 

= -L_xL«_«_ . (4.2) 



Using XY = Z"^ + we have T + 2uZ = X + k^Y ^ 2ky/XY > 2k\Z\, hence \Z\ < 2(k^\u\) - I* 
follows that max{X, k^Y} < T + 2uZ < jj^^^ , and in particular 

X,Y,\Z\ (4.3) 
Since Y > max{c^, p-}, X > v'^ max{a^, p-}, we also have ||7||oo ||7||, or more precisely: 

I I iji ll7l|V^ I I II7II iM ||7ll^'V^ /, ,N 

uy/c — |u| v^/k[k—\u\) fA/fc— |n| 

To compare the quantities ^{"f) and ^'(7), it will be important to show that they both lie in a 
certain Farey interval associate to 7. For ^{'j) we can use a geometric argument to determine this 
interval. Recall that ^{'y) = Re(7ti;). Looking at the images under 7 of the geodesies u — )■ w — )■ 00 
and — )• (x! — 7- — , if follows that 

$(7) G (7-", 700) n (70,7(^2/-")) := (4.5) 

(the endpoints of the intervals are not necessarily ordered increasingly). Since k'^Y > X, we can 
assume that cd{ck'^ + du){cu + d) 7^ at the expense of ignoring a finite number of matrices, which 
does not affect the pair correlation. To determine explicitly from (4.5) there are four cases to 
consider, and in each one we also define a Farey interval containing (using assumption (4.1) 
on u and assuming c > 0): 

L d > 0. Then jO = f|^) C (^, f ) =: J,. 

2. d < 0, cn + d > 0. Then ck^ + du > c + d > = (fg±^, f ) C (f±^, f ) =: J^. 

3. d < 0, ck^ + < 0. Then cu + d < c + d < = {\, ^) C (5^, =■, j^, 
A. d<0,ce + dn>0>cu + d. Then jO = (^, ^) C (f , 5^) =: J,. 

With this definition of J-y, we have the following asymptotic result for ^"(7). 

Lemma 3. Assume lo satisfies (4.1) and let^= ("^) GF with c > 0. Assume also that u, k'^ G Q. 
There exists Tq = Tq{uj) such that 

$(7), ^(7) G J-y, whenever T > Tq. 
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Proof. For '^(7) the statement was already proved (for all T). 
Using cZ — aY = —cu — d, dZ — bY = ck"^ + du, we infer 



a 



*(^) -- = " Y-r ' " ^ 

^(7) 



c Y — ct ' d Y — ex 



c + d Y - er 

We discuss only the second difference, since the analysis is similar for the others. We have 
l^^-j^l S> r^^/^, since the numerator is bounded from below as a result of the rationality as- 
sumption on u and k"^, and \d\ ^ II7II00 ^ T^^'^. Since the term involving is ^ T~^, it follows 
that the sign of ^'(7) — ^ is the same as that of j"'^" , and similarly for the other two cases, leading 
to the desired result about ^. □ 

Proposition 4. For each /3 G (^) l) have 

for some constants Ki,K2 > depending only on ^. The same equality holds with inter- 
changed. 

Proof. Let A/g^(^), respectively A/g ^(^), be defined as for TZq{^), with the additional condition 
min{||7||, ||7'||} > , and respectively ||7|| < . We trivially have 

^Q(0^AA>^(e) + 2AA<^(e). 

The estimate (4.2) shows that A/'(^_^(0 ^ T^qiC + KiQ^'^'^), the constant Ki being twice the 
implicit constant in (4.2). 

To show that A/g ^(^) = 0^{Q^~^^ log Q), we follow the same proof as that of Proposition 2 in [5]. 
Because of (4.4), at the expense of counting more pairs we can replace the set 9\q in the definition 
of A/q ^(^) with the set 

% := (7 G r : |7o;| < k, hlU ^ Q := ^^ |. 
L v^yk-\u\) 

Lemma 3 shows that ^'(7),<1>(7) lie between Farey fractions determined by 7 = ("^). More 
precisely, if D^g denotes the subset of 9\'q consisting of matrices with cd > 0, and let = (^, f). 
Then for each 7 G ^R'q we have that 

foij' (4.6) 

with 7' = 7 in Case 1, 7' = ( ^ ^+^) in Case 2, 7' = ( ^) in Case 3, and 7' = ( ^) in Case 4 
(see the four cases before the statement of the lemma). Note that in all four cases, ly C [0,k] or 
ly C [— A;,0], since ^{j) G {—k, k). Clearly each 7' G 9\q is associated with one, two, or three such 
pairs ^'(7), $(7) for 7 G 9^q. 

The proof now follows the same pattern as that of Proposition 2 in [5], using (4.6) above instead 
of (4.3) there, after further dividing D\q into the subsets of those 7 with C [— fe, 0], and of those 
7 with C [0,k]. For each subset the analysis of the associated Farey tessellation formed by the 
intervals is the same as in [5]. □ 
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5. The Farey tessellation and repulsion 

In the previous section we associated to each 7 G F an interval between two consecutive 
Farey points, such that $(7) S J^. We also associate to 7 the geodesic arc on the upper half plane 
connecting the endpoints of the Farey interval, which is part of the well known Farey tessellation. 
The purpose of this section is to quantify the statement that there is repulsion between $(7), ^{'y'), 
if the intervals associated to 7, 7' are disjoint.^ 

Lemma 5. Let 7 = G F, 7' = G F with f < j and c,d,f > 0. Then there exists 

A' G N such that 

K l\ , 



-1 0,'=^ 

Moreover, z/ maxjlCy, Ky/} ^ and we assume and k ^1, then 

Proof. Since the matrices 7 and -y's have the same first column, there exists K G 1^ such that 

e_ aK—b a 

f ~ cK-d ^ c- 



7( Q ^) = I's, which implies the desired equality. The fact that K > follows from j = > " 

A direct calculation provides 

K{c^{k^ + 2^2) + k'^df + cu{2d + k'^f)) + cd{l - k^) + u{k^df + c'^{l + - d^) 



Hi) - Hi) 



(A;2c2 + d^ + 2ucd){k'^p + + 2ucf) 



^ Ka{c,d,f) ^ Kk 



A 



where we denoted by a(c, d, /) the coefficient of K on the first line, and for the first inequality we 
used u ^ 0, /c ^ 1 and ^ > K ^ 1 > —^=^. If d ^ f, then we have a{c, d, f) ^ k^Yy, and if / ^ d, 
then a{c,d,f) ^ k'^Yy, which, together with max{Ky,yy} ^ Q^, proves the second inequality. □ 

For each > consider the finite set 

T{0 :=|J|m = 7i---7^:7, = (5;),i^ieN,i^i + --- + i^,^-ie|. (5.1) 

Lemma 6. Assume n ^ 0, /c ^ 1. Suppose 7 = ( « ^) , 7' = ( ) G SL2(Z), c, d, c',d' > 0, ^ ^ ^, 
and ^d) — $(7) ^ ^ for some Q ^ max{c, d, c', d'}. Then 

(i) 7' = 7M for some M G 

(ii) Furthermore, i/ M = 71 • • • 7^ is as in (5.1), then 

771 • I3 = ( q] ) , 

with . . . G {1,2, . . . ,Q}. 

Proof. As Q ^ max{c, d'} the fractions ^ and ^ belong to the set J-q of "extended" Farey fractions 
^ with (a, g) = 1 and 1 ^ g ^ Q. Let ^ = ^ < < . . . < ^ = ^ be the elements in Fn between 
I and ^. By Lemma 5 there are positive integers Ki and Kg^i such that 



— , . . . , — . (5.2) 

qi qoj \c dj V-1 0/' "' ' V c' d' J \qi qt^i) \ -1 0.'' 



Recall that two Farey arcs are nonintersecting, so the corresponding intervals are either disjoint or one contains 
the other. 
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The recursion relations for consecutive Farey fractions = j{ = i=2,...,£ — 1, 

£ ^ 2, (5.2), and the consequence $(7') — <5(7) ^ ^(-^1 + " " " + -^£+1) of Lemma 5 yield both (i) 
and (ii). Note that Ky, Yy ^ Q'^{k'^ + 1 + 2u) ^ under the assumptions on c, c', d', k, u. □ 

6. The case when 1{M) is large 

In this section we generalize Lemma 13 of [5]. Let S'Ijq{^), respectively '5^/q(0) denote the 
subsets of the set Sm,q{0 defined in (1.6) consisting of matrices with c, d > 0, respectively c > > 
d. Denote by A/'^q(^) the cardinality of <S^ q{S,). 

Lemma 7. Assume uj is in the half fundamental domain given by (3.2), and u^k"^ € Q. Suppose 
/3o S (|,l), M G r has nonnegative entries and max{X m ,Ym} ^ Q^^° . There exists Qo = Qo{(,,u}) 
independent on M such that q(0 — /^^ Q ^ Qo- 

Proof. We show that the region ilM,Qi(,) of {c,d) G (0,oo)^ for which 



|Sm(c, d)| ^ f^max{y^,y^A/} ^ Q^, 



contains no coprime integer lattice points. This plainly gives giO — 0- 

Suppose there is (c, d) G J7jvf^Q(^) n Z^. Write cw + (i = re*^, and let X = Xm, Y = Ym, 
Z = Zm and T = Tj\/ be given by (2.1). With Um = coth£(M) = 1 + 0(7^), the inequalities in 
the definition of Oa/^q(^) can be described as 

-2 ^mm<^ -2,-===— ■ —\. (6.1) 



e f/M + COS(0M - 20) Q2 ^/r2T:A2(f/M + COs(0M-2e)) 

Since sinO > O,cos0 > we can take 6 G (O, Denoting 6m = %^ — from the first and last 
fraction in (6.1) we have | sm25]\j\ ^ t^. Therefore 6m is close to 0, or to When 6m is close 
to we have | tan 5m | ^ \6m\ ^ I sin2JM| ^ f^- 

When (5m is close to it^ we similarly have |(5m =F fl ^ which we claim is impossible. From 

I tan (5a/ I _ |sin2(5M| , s 



it suffices to bound x+1os"21a/ from above, which would imply |tan(5M| thus contradicting 

I'^A/ T f I ^ f'- Next we consider the two cases that can occur. 

Case I: Z > uY. Since u,k'^ G Q, it follows from (2.3) that sin^M ^ Since cos 0, sin > and 
Om £ (0, vr), at least one of cos 26m + cos 6m or cos 26m — cos 9m must ne nonnegative, showing 
that cos 26m ^ — | cos 9m \ , and so we have 



1 + cos2(5m ^ 1 - I cos 6*4/1 = 1 - \/l - sin^ 9m > ^• 

Since Um — 1 ^ it follows that i^^'^^l ^ 1, proving the claim. 

Case II: Z < uY . In this case sin0j\f < 0. We also have cos0j\/ > by (2.3), since 

Ay - T = 2uZ -X + iv"^ - u^)Y >uZ + {v"^ - u^)Y - jY > 

for Q large enough, where we used XY — Z"^ = v'^, and v > u for to in the region given by (3.2). In 
this case 6m £ ( — X' ^) ' have to show that \6m + f | ^ 7; leads to contradiction. 
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From |sin2(5A/| < 7. we have \ tan 6m\ > T, and (6.2) gives 1 > Tjj^^^^^. This leads (for 
large T) to ^ » Um — 1 » ^(1 + cos26m), and therefore to Um + cos 25m ^ Back to (6.2), 
we infer | sin 26m\ ^ ^ , leading in turn to 



< I tan 5m I 



tan 



1 + 



< 



1 + ^ 



cu+d 



tan 



1 + 



cu+d 



tan(%i)|' 



where we used 9m & {-^,0). This further gives 



1 + 



cv 



;tan (^m/2) 



cn + d 

and so because u, /c^ G O we infer 



1 + 



cv 



CU + d\v\Y 



Z 

— — u 



+ 



TYJ 



2^2 ' 



1 <^ |(cu + d)Y + c(Z - uY)\ <^ (cn + (i)y 



1 1 



(6.3) 



As |u| < 1, (6.3) and (4.3) now yield cu + T ^ Q^'^", which contradicts cu + d Q. 



We have thus shown that | tan 5m I ^ ^- Next we consider two cases, tanS < v and tan^ > 



V. 



Case (A): tan^ < v. Recall that ^'(M) = u + f tan (%^), and we also have n + utan( 
Since |5m| ^ ^, tan (%^) is also bounded, leading to 

ck"^ + du 
cu + d 

Since ^'(M) < 1, we have Z <^Y, and from XY - Z"^ 



|tan (6'm/2) - tan 6*1 = v\ tan^A/l |l + tan6'tan {6m/2) \ < 



ck^+du 
cu+d 



'Wo 



follows that |^(M) - #(M)| <C Y^^ < Q^'^po, so 



\z_ 

Y 



ck^+du I 



we conclude X <^ Y. From (4.2) it 
< On the other hand. 



A Z D + uC 1 ^_2«„ 
c y CY Y^ 

where we assumed without loss of generality C ^ D {li D ^ C , then use ^ instead of ^). We 
conclude that 

A ck"^ + du 
C cu + d 



< Q 



-Wo 



(6.4) 



If nonzero, the left hand side of (6.4) is ^ c(Ja+d) ' ^sing the rationality assumption u, A;^ G Q. 
From tan 6* < -y and n < 1 it follows that c ^ d, so C{cu + d) <^ V d'^Y < ^jY^j^i <^ Q, obtaining 



a contradiction. It remains that ^ 



ck~+du 



Mc+Nd ^i^i^ M,N,P,R G N constants, so d > 



cu+d Pc+Rd 

Pc + Rd;^ C VY (using D). It follows that > Y^m > d'^Y » ^ Q^*, which again 
provides a contradiction. 

Case (B): tan^ > v. We have | < cot6' < i, and since |5m| ^ ^ it follows that cot (^) is 
bounded as well. Consequently 



1 



cu + d 



^(M) cfc2 + d^/ 



v\ tan 5m I 



|l + cot(g)cot(gM/2)| 
I (li cot (61) + t;)(ncot(6lM/2) + v)\ 



-Wo 



In this case ^'(M) > 1 implies Z > so X > Z > 1", and from XY - 
Z ^ Q'^". Taking into account (4.3) we arrive at 



v"^ we have 
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Assuming A > B (the other case is similar), we infer 



Y C D + uC VY 1 ^_2A, 
Z A AZ yOCZ X 

and thus |^ — ^^"^^^ | ^ Q~^^° . This leads to a contradiction for large Q as before. □ 
7. Approximating the number of lattice points in planar regions by volumes 

In this section we approximate Mm,q{C) foi' G S by volumes of three dimensional regions, 
where 6 C F is the set of matrices with nonnegative entries, distinct from the identity. Using the 
result of the previous section, we also show that the sum of A/j^ q (^ ) over subsets of M 6 S can 
be approximated by the corresponding sum of volumes. 

To count points in two dimensional regions we use Lemma 12 of [5]. The prototype for its 
application in the present setting is given in the following simpler counting problem. By well 
known asymptotics for the number of points in expanding hyperbolic balls we have Sg* ~ (for 
the notation see Section 3). In the next lemma we show that in half balls we have half this number 
of points. 

Lemma 8. Let Bq = i^yiq be as defined in Section 3, with k ^ 1. Then 

BQ = ^ + 0,{Q'y<^+^), 

and so Bq ~ 

Proof. Replacing b = "'^"^ , the condition |7a;| < k is equivalent to 

,9 o? \ , n, ,9 , , 2ad — 1 2au ,„ , 

- ^]{c^k^ + d^ + 2cdu) + ^ — + > 0. (7.1 

/ c 

A direct calculation shows that ||7|| ^ Q is equivalent to 

,9 2au\ , 9,9 ,9 , , 2du 1 — 2ad ( 2a\ ^9 ,„ , 

A;^ + — (c^fc^ + + 2cdu) + + ^ — + ( U^^kQ ■ (7.2) 

Fix a = yI and let 7 = ( " ^) G with c > 0. The contribution of the matrices 7 with |c| < 
or \d\ < to the error term is ^ Q^"*"", so we can assume c > and \d\ > 

We show that the matrices 7 with ^ > k'^ contribute negligibly to Bq. From (7.1) and c^k^ + 
+ 2cdu ^ c^u^, it follows that for such 7 we have < ^ - fc^ < ^(^ + ^) < Q^""^", so 
G [kjk + m] with m <^ y^fc^ + KQ^~^°' — k <C Q^~^". Since \a\,c <^ Q from the equidistribution 
of the Farey fractions J-q in intervals / of length |/| > with 5 = 2a — 1 G (0, 1) it follows that 
the number of pairs (a, c) is <C Q^"^" as long as a G (5,1)- Since the number of values d can take 
is ^ ^ < Q^~", there are <C Q^~^°' such matrices as long as ^ > so they can be absorbed in 
the error term. 

Therefore we can assume |a| ^ kc, and the condition \'yu}\ < k is satisfied except for a negligible 
number of matrices. Via (7.2), the condition ||7|| ^ Q can be replaced without affecting the 
asymptotics by 

^2 ^ a2 _ 2mx\ ^ ^2 ^ 2cdn) ^ Q'. 
c^ c J 

Therefore we can apply Lemma 11 in [5], with L = |, to the set 



(a, d) G [-kc, kc] x [-Q, Q] : {k^c^ + - 2uca){k^c^ + d^ + 2ucd) < Q^c^j 
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(recall Q = — 9^ ), and conclude that 



c=l 

Mobius summation and a change of variables a = cz,d = Qy, c = Qx then gives 

i?Q = ^Voi(yQ) + o,(Q"/6+0, 

where Vq denotes the set of points {x, y, z) G [O, -^j x [ — ^5 ^] x [— fc] such that (k^x^ + + 
2uxy){k'^ + — 2uz) ^ 1. The substitution xu) + y = re*^, z = viai\t + n, then yields 



./3/2 .£2^ 2 

Vol(yQ)= / / / drdedt = —, 

J 812^7,12 Jo Jo cosH 2A 



'/3/2-7r/2J0 JO 

which concludes the proof. □ 

Next we seek to replace inequalities defining the set Sm,q{0 ™ (1-6) with simpler ones, involving 
only the entries (a, c, d) or (6, c, d) of the matrix 7 = ( " ^ ) • Using 

XyM = o^Xm + h^YM + 2abZM, Y^m = c^Xm + d'^Yu + 2cdZM, 
Z^M = QcXm + bdYM + {ad + bc)ZM, 

and substituting b = , we find 

hMf = f + ^ - 2.^) y,M + yM + 2inc-a)idYM + cZM) ^ 3^ 



I fl/r I / I. ^ / ^2 a^A^ , 2a{dYM + cZm) - Ym ^ ^ 
|7Ma;| < k [k ^ ) Y-yM H 2 > ^• 



The previous formulas lead us to consider the cardinality Mm,q{C) of the set Sm,q{0 of integer 
triples {a,c,d) satisfying (1.7). Let 5^g(^), respectively be the subsets of Sm,q{0 for 

which c,d > 0, respectively c> > d. 

Lemma 9. Assume u, k^ G Q. 

(i) There is a constant K = A'(^) > such that, for every Q, the number of pairs (7,7') G 
with II7II, ||7'|| ^ Q, min{|7a;|, |7'u;|} < A: < or max{|7a;|, |7'a;|} > k > and 

1^(7') -^(7)K^, (7.4) 

is at most K . 

(ii) For each a G (0, 1) the following asymptotic estimates hold: 

E ^tid^) ^ E ^M.Q(i+o(Q-^/^))(0 + 0{Q'^-), (7.5) 
Me6 Afe6 

E ^M.q(0 ^ E ^tQ(l+0(Q~^/^))(^) +o(Q^+") 

M66 A/G6 

(iii) For M £ & we have individually 

^rM,Q{0^^fM,Qii+o(Q-^))i0+O{Q'+n, A^a/,q(0 ^ A/"m,Q{1+O(Q-))(0 + 0(Q^+"). 



16 



FLORIN P. BOCA, ALEXANDRU A. POPA, ALEXANDRU ZAHARESCU 



Proof, (i) Assume first ^ > /c > |7a;| or ^ < A: < |7f^|. Since ^(7) is the x-intercept of the 
geodesies from a; to jlo, it follows that \^oj\ < k if and only if |^'(7)| < k, so we have (assuming 
a > 0, the other case being similar with k replaced by —k below) 



< 



^(7) 



< 



cl>(7) 



+ ^(7)-«>(7) < ^, 



by (4.2) and I $(7) 



1 \u+d/c\ 1 



Since A: G Q, there are finitely many such pairs 



(a, c), and repeating the argument with - replaced by -j we obtain that there are finitely many such 



matrices 7 (also using the fact that there are finitely many 7 with k between ^ and From (7.4) 
and the fact that $(7) G Q, there are also finitely many matrices 7' satisfying the assumptions. 

Finally assume ^ > A; > |7'ci;|, the remaining case being similar. Then |$(7')| < A; < and as 
before we have (assuming a > 0, otherwise replace k by —k) 



k 



< 



< 



$(7) 



+ 1^(7) - $(7')! < 



,2 ' 



and we conclude as in the previous case. 

(ii) Next we look at pairs (7,7' = 7M) satisfying (7.4), with \a\ ^ kc, \\^\\, ||7M|| ^ Q, estimating 
their contribution to the left-hand side of (7.5) according to whether Yy < Q^" ov Yy ^ Q^". By 
part (i) we can assume <^ Y^, and by (7.3) we have ||7|| ^ Q'^ or ||7|| » Q'^, respectively in 
the two cases. 

Assume first Y^ < Q^"- With the Farey interval J-y associated to 7 defined before Lemma 3, the 
Farey intervals and J^m are either disjoint or one contains the other. Since each Farey interval 
is associated with at most three matrices 7 S F, it follows as in the proof of [5, Proposition 2] that 
the number of pairs (7,7M) is ^ Q^^" InQ. 

Therefore we are left to consider pairs (7,7M) with Yy ^ Q^a^ |7a;|, |7Ma;| ^ k, and with 
||7||,||7M|| ^ Q. These conditions are satisfied by 7 in either 5m,q(0 or Sm,q{0^ as we can 
assume <^ Y^, X^m ^ y-fM by part (i). Without loss of generality we assume c > (i > 
(otherwise substitute a in terms of 6, c, d in the left side of (7.3)), and show that 



\{2a-uc){dYM + cZM) 



c 



(7.6) 



The inequality ||7M|| ^ Q plainly gives Ym ^ Employing also {dYM + cZu)'^ + v'^c^ = Y^mYm 
we arrive at (7.6). By (7.3) the claim follows. 

(iii) By the proof of part (ii), it remains to consider pairs (7,7M) with Y^ ^ Q", |7w|, |7Ma;| ^ A;, 
and with (i < 0. Without loss of generality we can assume c > (otherwise substitute a in terms 
of 6, c, d in the left side of (7.3)), and (7.6) follows trivially since M is fixed, with the upper bound 
being now Q~°' . □ 



Lemma 10. For any M G 6, uniformly in M and ^, 



Yol{SM,i) + 0,{Q 



U/6+e\ 



Proof. Mm^q{S,) represents the sum over c G {1, . . . , Q} of the number of integer lattice points (a, d) 
with ad = 1 (mod c) in the region il. = i^M,Q,c{0 of points {a,d) G [—kc,kc] x [—Q,Q] for which 
\EM{c,d)\ ^ ^ and {l + ^ -2u^) nmx{k'^c'^ + d'^ + 2cdu,c^XM + d'^YM + 2cdZM} ^ Q"^- Applying 
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Lemma 11 of [5] with q = c, Area(r2) ^ cQ, length((9il) ^ Q, and L = c^/^, we find 

^m,q{0 = Area (J^m,q,c(0) + Oe(Qc-V6+-) j . 

Mobius summation applied to the function h{c) = ^ ATea{Q,M,Q,c{0) with ||/i||oo ^ Q, and the 
change of variables {c,u,v) = {Qx,Qxz,Qy), yield 

~ ^ fir , 

= ^Vo1(5m,5) + 0,(Q"/6+^). 

□ 

When restricting to the subset S^.jq{Oi have the following better estimate. 

Lemma 11. Let 6' C & be any subset. For each /3o G (|, l) the following estimate holds (uniformly 
in ^ on compacts) 

Me6' Me6' 

Proof. The contribution of c ^ d to A/'^q(^) is 

A/'m.qCO = E ^K"'*^) ^ ^ ^M,Q,c(0, fl'i = 1 (mod c)}, 

cs;Q/{t^yxlF) 



where = 0'^,^ g ^(^) denotes the set of points (a, d) G [— A;c, kc] x [c, Q] with 

max{A:^c^ + (i^ + 2c(iti, c^Xa/ + (I'^Ym + 2c(iZA/} ^ — — Q^, |Sm(c, d)\ ^ 2 • 

Applying Lemma 11 of [51 to U with Area(J^) ^ lengthfSr^) < cH — ^ ^ ^ ^ ^S=, 

L = c^/e ^ c ^ d ^ we find 

^AwiO = E Area {n\,^QM + (^^^) ) • (7-8) 

A choice of the pair (7711,7712) where ttt-i = max{A,B}, m2 = max{ C,-D} determines M uniquely, 
and max{i;^C^, p-D^} < Ym, with similar inequalities for A,B,Xm- According to Lemma 7 we 
should sum over M with min{XM,yA/} < Q'^'^°. Thus the total contribution of the error term in 
(7.8) to (7.7) is 

« E — E c-v^+^«giogQ (—T'^^«sQ^ 

^-^ 7712 \n^i J 



)(ll+/3o)/6+3e 



Mobius summation over c applied to the function h{c) = ^ Area(r2M,Q,c(0) with ||/7||oo ^ v^c^D^ ' 
the change of variables (c,a,d) = (Qx,Qxz,Qy), (x,y,z) G [0, — , 1 x [0, — 1 x [— A;,A;], 
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and Lemma 7 provide 



Xm,Ym<:Q^^'> 

''^ Afe6 

where 5"^/^ denotes the subset of S^j ^ with the additional condition x ^ y. 

The contribution of d < c to J\f^j q (0 > denoted Af'lj q (C) > is estimated similarly, by summing first 
over d ^ yjy ' ^^^'^ o^^'^ pairs (6, c) with be = —1 (mod d). We obtain the same equation as (7.9) 
with S']^,j ^ replaced by S'lj ^ = S'^j ^ \ 5^/^, thus proving (7.7). □ 

Corollary 12. For each /3o G the following estimate holds: 

E ^M,QiO = ^ E Vol(5+ ^) +O,(Q("+/^0)/6). 
Me6 Alee 
Proof. This follows from Lemma 11 and estimate (7.21) in [5]. □ 



8. Extra symmetries for uj = i 

In this section we assume u = i and make use of extra symmetries of the hyperbolic lattice 
centered at i. For each matrix M = (^^) GF consider the cardinality Mm,q{0 of the set 5m,q(C) 
defined in (1.6). We first show that in the expression (1.5) we can restrict ourselves to matrices M 
having positive entries, and then we show that except for finitely many such matrices M we can 
restrict ourselves to counting only elements 7 G 5Af,Q(0 with positive elements in the second row. 
Thus we can make use of results of the previous sections to estimate the quantity TZq (^ ) . 

Recall the set S of matrices M ^ I with nonnegative entries, and denote s = {i~q)- The signs 
of the entries of sM, Ms, and sMs show that F \ {/, s} is partitioned into & U s& U &s U s&s. The 
equalities ^{gs) = ^{g), si = i, and \\g\\ = \\gs\\ yield 

<SMs,QiO = <SM,Qi(,), <SsM,QiO = <Sm,q{(,)s, 

AAm,q(0 = ^Ms,q{0 = a/;m,q(6 = A/;ms,q(0- 

Therefore (1.5) becomes 

7^*(0 = 2 5] aAm,q(0. 

Mee 

Let 5^q(.^), respectively S^,jq{£,), denote the subsets of Sm,q{0 consisting of matrices with 
c,d>0, respectively c > > d. Denote by J^mq(0 the cardinality of 5^/q(0- 
Recall the finite set J-{^) defined in (5.1) and let 
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Lemma 13. (i) For any M £ & the mapping 7 1— )• ^yMs ^ defines a bijection between the sets 



and 

(ii) // M G 6 \ then 5,", q(0 = SJj*Q{0, so that A/^/.qIO = ^t^giO- 

Proof, (i) Let 7 = ^) G and jMs'^ = il})- On one hand c>Oandf<^<^ 

imply c = -cB - dD > 0,d = cA + dC > 0, while c > and J > imply c = cC + dD > 
and d = -cA - dB < 0. On the other hand, utilizing also ||7|| = ||7M||, sM^^s"^ = M*, 
||7Ms~^M*|| = II7II, we conclude that the map above is a bijection. 

(ii) Suppose, by contradiction, that there exists 7 € ^j^f q(0 \ '^M*QiC)- Setting 

- ( aB+bD \ _ (a' b' \ 

1^^^ — \ cA+dC cB+dD ) ~ \ c' d' ) ~l ' 

notice that Q ^ max{c, — d, |c'|, and two cases can occur: 

1) =i <% Then c' > 0, (i' > 0, and ^ < ^ f < 5^. By Lemma 6, 75-^ = 7'Mo with 
Mo G J"(0, so M = (Mos)~\ contradiction. 

2) § < ^ < Then c' < 0, d' < 0, and f < 5^ ^ < f^. Lemma 6 gives -7' = 7S-IM0 
with Mo G -7^(0) so M = -s'^Mq, contradiction. □ 

Remark 14. The analogue of Lemma 13 (ii) holds for S^j q{£,) in place of '5^/q(C) (see the 
notation preceding Lemma 9). Namely there is no matrix 7 = ("^) G q{^) with either 
~c^^°^U^"7^' Ii^deed, referring to the notation in the proof of Lemma 13 (ii), we have 
c'^ + d'"^ = c^Xm + d'^YM + IcdZM ^ Q^, so that Q ^ max{c, — d, |c'|, and the rest of the proof 
goes through unchanged. 

Consider now the region Sm,s, in (1-8), which in the present case u = i becomes the region of 
triples {x,y,z) G [0,1] x [-1,1]^ for which^ 

I'^Mix, y)\ ^ max{x^ + y'^^x^Xu + V^Ym + 2xyZM] ^ , n • (8.1) 

1 + 

Consider also the subsets •S'^/^, 5'^/^) S'^f(_ of Sm,£, defined, respectively, by y > 0, y < 0, y < and 

Vol(S-'* ) = Vol(5+ , VM G e \ ^{0- (8.2) 

Lemma 15. If M G S \ then Vol(5^^_^) = Vol(5j^:/* ). 

Proof. Suppose by contradiction that the set S]^j ^ \ S]^f^ contains an interior point {xq, yo, zq), that 
is {xo,yo,zo) G (0,1) x (-1,0) x (-1,1) satisfies both (8.1) and ^ G (O, f ) U (^,00). Now the 
set of rational points 

S^Q = I ^, : (c, (i) = 1, Q^c>0>d^ -Q, ad= 1 (mod c), \a\ < c| 

is dense in D = [0,1] x [—1,0] x [—1,1]; indeed for each parallelepiped R C D, we can count 
the number of points in the scaled sets QR D Qi^q, as in the proof of Lemma 8, and conclude 
that Qq n i? is dense in R. Therefore for large enough Q, we can find points in fig arbitrarily 



— < ^ < ^. As in Lemma 13, the mapping {x, y) 1— )• (x, y)Ms ^ = (x, y) defines a diffeomorphism 
between the sets ST^/t. and Stjt e, showing in particular that 



■^As Xm, Ym ^ 1, Zm ^ 0, when y > the inequahty + ^ TT^^ obsolete. 
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close to {xqjUqjXqZq), and it follows that there exists {a,c,d) € 5a/,q(0 with c > > d and 
^ € (O, U (^,00), which contradicts Remark 14. □ 

Estimates for A/j^q(^) with M ^ J'iO are derived from those on q(0 t>y Lemma 13. We 
can now estimate X^A-zee-^Af.QlC) by first breaking the sum into sums over and over 6\T{S,); 
for the first sum we use Lemma 10, while for the second we use -f^M q{0 = -^m* q(0 ^^'^ Lemmas 
9 and 11. Finally, employing (8.2) and Lemma 15, we find 

E ^^^^Q(0 = ^ E Vo1(Sm,c) + 0^(Q(^^+*)/^). (8.3) 
Me6 ^ Me6 

To complete the sum to M G T, note that Vol(5jv/,5) = VoI{Sms,^)- This is also seen to coincide 
with Vo1(5<ja/,5)i and thus with VoI{SsMs,£,), by employing EgMiu, —x) = ^m{x, y) and the change 
of variable (x, y) 1— )■ {—y,x) if (a;,y) G [0,1] x [—1,0], respectively {x,y) 1— (y, — x) 1— )• {y,—x) if 
(x,y) G [0, 1] X [0, 1]. This proves (1.9). 

9. A CLOSED FORM FORMULA FOR Vo1(5a/_5) 

In this section we evaluate the volume of the body Sm,s, in (1-8) for arbitrary co, which leads to 
the formula in Conjecture 1. For to = i, the proof of this conjecture is based on the results of the 
previous section. 

The volume can be brought in closed form using the substitution 

xu + y = re^^ , z = vta,nt + u, (9.1) 

with E]\f{x,y) given by the first equation in (2.4), /c^x^ + y^ + 2uxy = r^, x^Xj\,f + y^^Af + 2xyZjv/ = 
r2sinh^(M)(coth£(M) +cos(0M -20)), k'^ + - 2uz = to 

Vo1(5m,5) = W BMiC,t)^- = v BMi^,t)^-, (9.2) 

yarctan((-fc-n)/t)) COS t J/3/2-7r/2 COS t 

with (3 G (0,7r) such that co = ke^^ , where B]\f{^,t) is the area of the region defined in polar 
coordinates (r, 6) by 

'^(sin0,fesin(/3-0))G [0,^] x [-§,§] 

' V |sin(gM-2g)| ^ 2^cos2t . r 1 1 M 

r mm s i > 

U t^A/ + cos(0M - 20) I 'sinh^(M)(C/M + cos(0Af-20))/' 

with Um = coth£(M) = > 1, T = ||Mf . 

Using the second condition in (9.3), we have r'^^^^ k(k-\u\) • Hence the first condition in 
(9.3) can be replaced by ^ 6* ^ vr, and the area can be expressed in closed form: 

^ min C/m + cos(0Af -29)}- f | sin(0M - 20)| 



2u Jo Um + cos{9m - 26) 

with /+ = max{/, 0}. 

Since we are interested in the pair correlation of the angles 0^, we define 

n%{i) := # |(7,7') G : 7 / 7', ^ 0(7) - 0(7') < ^} • 

Following the approximation arguments from Section 8 of [5], from (8.3) we obtain the following 
asymptotics: 
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Proposition 16. For 00 



i the estimate 



2C(2) 



M&r\{i} 



holds, where 



13/2 



//3/2-7r/2 

Since Ba,/(j^, i) = -Ba/(C, 0) cos^ t, we find 



,t 



2 cos^ t ' / cos^ t 



dt 



Bm{0 = Bm["4,^Y-^. 



Taking derivatives we obtain 

B'm{0 = ^J 



sin(0M-20)| 



Um + cos( 



26) 



de, 



with I^^M = {0 e [0, tt] : | sm{eM - 26')| ^ |; min{C/A/ + cos( 
With Cm ■= = tanh 



29), 



1 



sinh£(Af) 



}} (recah A = 2t;2). 



G (0,1) we have sinh^(M) 



-, cosh^(M) 



f^M — Cm 



sinh 



' l(M) ' 



2u 



sinh^(Af) ' "^^^^^M 2 7 2u 

[— 7r,7r] the integrand is even on [— 7r,7r], and so we have 



. Using the change of variable u = 26 — 6m ^ 



B'Mm 



TT 



2A2^2 



Um + cos u 



du, 



with J^^M = [J^^Ij U J^%) n [0,7r], where 



(2) 



J, 



(1) 



n : cosn ^ —Cm, sinn ^ 



sinh^(M) 



Jt^jj/ = • cosn ^ —Cm, sinn ^ C{Um + cosn)}. 



A direct calculation provides 



' [0, arccos(-CA./)] if C 5* sinh£(M) = v^I^ 



J. 



(1) 



arcsm 



smh£(A/) 



)]u[vr 



arcsm 



sinh£(M) 



arccos 



(-Ca/)] 



if 2 sinh 



arcsm 



0] 



if^^2sinh^^ 



2-; ^ C 5^ sinh£(M), 



J, 



(2) 



sinh£(M);j S ^oiiiix 2 

' [arccos (-CAf),7r] if ^ ^ sinh£(M), 
arccos(— Cj./ ), a + arcsin(C/A./ sin a)] U [vr + a — arcsin(C/A/ sina),7r] 

if 2 sinh (^) ^ e ^ sinh£(M), 
Jvr + a — arcsin(C/A./ sin a), vr] if ^ ^ 2 sinh (^^^) • 

'[0,^] if ^ ^ sinh^(M), 

)] U [vr - arcsin (^^5777^0) , a + arcsin([/A/ sin a)] 



, arcsin 



^ [0, arcsin 



sinh i{M) / 



U[-K + a — arcsin ([/a./ sin a), vr] if 2 sinh (^) ^ ^ sinh£(M), 



^ sinh £(A/) 



)] U [vr + Q — arcsin(J7A/ sin a), vr] if ^ ^ 2 sinh (^^^) , 
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where a = a(^) = arcsin (— ^^==) G (O, |). We obtain 



with f({£) the function defined in the introduction. Letting R2{0 = li™Q-5.oo ^^q(0> from 
Proposition 16 we infer 

(note that /^(O) = so we can include I in the range of summation). Taking into account that 
the pair correlation distribution -^2(0 in the introduction involves normalized angles, and that 
Bq ~ ^Q^, we have 

3C\ A„g^^^ _ f3C\ nA^dR?. 



This leads to the formula for g2 stated in Conjecture 1, and proves Theorem 1 when oj = i. 

10. The case u) = p 

In the case of the other elliptic point oj = p one can take advantage of other symmetries to prove 
Conjecture 1. Consider the matrix w = fixing the point p. This time we partition the upper 

half plane EI into three regions which are permuted clockwise by w: 

I = {z G M : Rez > i, |z - 1| < 1} , II = {z G M : Rez < i, |z| < l} , 

III = {z G M : |z - 1| > l,\z\ > 1}. 




Figure 2. Symmetric geodesies through p 



The condition that jp belongs to one of the three regions is easily stated in terms of {X, Y, Z) 
given by (2.1), using the relations Re{'yp) = y, \ jp\'^ = y, \^p - Ip = 1 + : 

jp£l ^2Z > X, 2Z >Y, 7p G II ^ y > X, y > 2Z, 

7P G III ^ X >Y,X > 2Z. 

Next we determine the restrictions that the condition jp £ I places on the entries of 7. As a 
consequence of 2Z > Y, 2Z > X, a quick check shows that the entries of 7 are nonzero. Since 
abed = hc+ (bc)^ > 0, we also have that ac and bd have the same sign. In fact ac > 0, bd > 0: if 
the contrary were true, from 2Z = 2{ac + bd) + 2ad+ 1 > it would follow ad > and without loss 
of generality we can assume a > 0, c < 0, d > 0, 6 < 0; from ad > —ac — bd it would then follow 
that d > — c, a > —b, which implies ad — be > 1, a contradiction. Since ae > 0, bd > 0, among the 
matrices 7, 'yw, "jnP' with the same coordinates {X, Y, Z) precisely one has entries of the same sign. 
Therefore we assume from now on that 7 has positive entries whenever 7/3 G I. 
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Using the substitution b = ^^^^-^ , one checks that 

2Z >Y 2a -c> ^ '^'^t^ , 2a > c 

+ d'^ + cd 

(the last equivalence follows since the fraction is less than 1 and 2a — c is integral). Similarly 
2Z > X if and only if 2d > b. In conclusion, the point 'jp belongs to I if and only if, after perhaps 
replacing 7 by or 7^^, we have 

a,b,c,d>0, ->l, ^<2. (10.1) 
c 2 d 

Since w permutes the regions I, II and III and wp = p, the previous discussion shows that T 
can be partitioned as 

r\{i,w,w'^}= \J w''mw', 

r,se{0,l,2} 

where 

We can now rewrite the sum (1.5) as follows. Since \\gw\\ = \\g\\ and ^{gw) = ^{g), we have 
'5mw»,q(0 = '5m,q(0- Moreover the map g ^ gw~'' takes SM,QiO bijectively onto 5^'-a/,q(0i so 
^fw^Mw^Qi^,) =J^M,QiO- We infer 

To state the equivalent of Lemma 13, we further divide region I in two regions Ii and I2, 
according as \z\ < 1 or |z| > 1 (see Figure 2). Lemma 2 shows that jp i—?- 7^0 is a bijection of Ii 
onto I2. Let also SOTi, respectively TI2, denote the subset of M G 9K with Mp G Ii, respectively 
Mp e I2. 

For M = (^ g ) G let 5^/'q(0i respectively S]^^q{(,) be the sets of those 7 = ( « ^) G q(0 
for which § <-i< respectively ^ < < With as in in (5.1) define 

Lemma 17. (i) The map 7 1— t- 'yMw^^ is a bijection between 5jv/'q(C) o.iT'd •S^]\^-i^2 q{0> ^"^d the 
map 7 I—)- "fMw is a bijection between 5^/'q(0 '^^'^ '^w^m-'^w q('^)' 
(ii) IfMeTl\ then S]^,q{0 = 5j,,;q(0 U S]^J%{0^ so that 

The proof is very similar to that of Lemma 13 and we leave it as an exercise for the reader. 
Let now 9?tj(^) = i G {0, 1}, and define the sets 

61(e) = {wM~^w^,w'^M~^w'^ : M G 9?Ti(C)}, ©2(6 = {w'^ M-^w,wM-^w : M G 97t2(C)}, 

both easily checked to be contained in & (namely they contain matrices with positive entries). 
From Lemma 17 and the fact that Mmw^,q = J^Mw-^ it follows that: 

E-^^^.q(^)= E ^M,Q{i)+ E ■^m,q(^)+ E ^tid^) 

Mam MemnHO Mei»ti{0uiOT2(0 M66iK)u62{C) 
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Since now we only sum over matrices with positive entries, the approximation arguments em- 
ployed in the case uj = i also apply here, with defined in (1.8) and ^ the subset of Sm,^ 
defined by the additional condition y > or y < 0, leading to 

# E -^^AQie) ~ E Vo1(5m,^) + E Vol(5+ ^) + E Vol(5+_^). 

Using equalities analogous to those of Lemma 17, for volumes instead of the number of lattice 
points, we arrive at 

^$(0 ~ ^ E Vo1(Sm,5). (10.2) 

Finally the sum of volumes in (10.2) can be extended from 9Jt to F since Vol(S'^rA,/^s^^) = 
\o\{Sm,{)- To check this, we use the polar coordinates xp + y = re*^ from (9.1), leading to formula 
(9.2) for YoI{Sm,s,)- Note that the inequalities defining the volume in polar coordinates only depend 
on i{M), r and 6m — 20, with the restriction 9 £ [0,7r]. Since i{M), 9m only depend on Mp, it 
follows that yol(SMw'>,() = Vol(5A/^g). 

To show Vol{S^M,() = Vo1(S'a/,5), let ^jx^y G SL2(M) be any matrix with lower row {x, y) / (0, 0); 
note that j{'yx,y, p) '■= xp + y = re*^ in polar coordinates. The transformation (x, y, z) i— )• {x' , y' , z) 
with {x\y') defined by -ix',y' = lx,yW~^ has x' p + y' = j{'yx,y, p)j{w~^ , p) = re*^^"''/^), so in 
polar coordinates it corresponds to {r,6) i— ?• (r' = r,9' = 9 — tt/3). Since M i— ?• wM results in 
9m ^ 6m — 2vr/3, and the inequalities (9.3) defining Sm,£, involve only 6m — 20, it follows that the 
transformation (x, y, z) i— )■ {x' , y' , z) above maps the volume Sm,( onto a volume S'^j^j ^, defined like 
SwM,^ but with the range 6 £ [0, tt] replaced by 9' € [—ir/S, 27r/3]. Since in the formula for Bm{Cj t) 
following (9.3) the integrand has period vr, we conclude Vo1(5^m,5) = Vo1(S'm,5), 

This concludes the proof of (1.9). The formula in Theorem 1 follows from the results of Section 

9. 



Appendix A. Arithmetic description of closed geodesics through p 

In this appendix we discuss the connection between the hyperbolic lattice centered at p and closed 
geodesics on the modular surface passing through n(p) where 11 : IE — >■ H/F is the projection map. 
In the case of the hyperbolic lattice centered at i, the corresponding geodesics are the reciprocal 
geodesics studied by Pricke and Klein [5, Section 2]. We similarly describe the primitive closed 
geodesics passing through n(p), which have an interesting arithmetic structure. 

Closed geodesics on the modular surface correspond to conjugacy classes {g} of hyperbolic ele- 
ments 5 € F. If C H is the axis of g (the semicircle connecting the two fixed points of g on the 
real axis), then the geodesic corresponding to 5 on X is n(2;o — >• gzo) for any fixed point zq G ag. 
We are interested in geodesics passing through Il{p). Let TZ denote the set of conjugacy classes of 
hyperbolic elements which contain a matrix g whose axis passes through p. Let TZ"^""^ C TZhe the 
subset of primitive conjugacy classes. We will give an arithmetic description of TZ^^^^. 

Let <^=(^^)bea primitive hyperbolic matrix whose axis Ug passes through p. The fixed 
points A > A of 5 satisfy the equation 

CA2 + (D-A)\-B = 0. 
Imposing the condition (A — A)^ = \p — M'^ + |p ~ M'^ conclude that 



p G flg ^ D - A = 2{B -C). 



(A.l) 
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The matrices wgw^, and w'^gw are also primitive. Their axes are the same as rotated by ib27r/3 
around p, hence the class {g} contains a matrix, still denoted by g, with gp £ I (the first region 
in Section 10). This is equivalent to A G and Iie{gp) > ^, which is further equivalent with 

A,B,C,D being all positive or all negative. 

In conclusion each class {h} E t^p"™ contains a matrix g as above with positive entries satisfying 
(A.l), and so we are left to describe the set of such matrices and determine when two such matrices 
are conjugate. 

The condition (A.l), together with AD- BC = 1, implies that {A + Df - 4:{B'^ + C'^ - BC) = 4. 
Writing k = gcd{B,C), B = kBo, C = kCo, T := A + D, the pair {T,k) is a solution to Pell's 
equation 

- Ak'^A = 4, (A.2) 

with A = Bq + Cq — BqCq. In fact, {T,k) is the minimal positive solutions since g is primitive. 
Direct computation using (A.l) shows that 

coshd{p,gp) = — - 1. (A.3) 

We are led to define the set 

Vp := {{Bo, Co) : Bo, Co > 0, (Bq, Co) = 1, A = + " ^oCo not a square } 

= U ^A, (^-4) 

where Pa is the finite subset of pairs {Bo,Co) as above having fixed A. We denoted by "Dp the 
set of possible such A, which is the same as the set of positive numbers all of whose prime factors 
are congruent to 1 mod 3, or the prime 3 appearing to the first power. The cardinality of Pa is 
2^+*^ with z/ the number of distinct prime factors p = 1 (mod 3) of A. We conclude that there is a 
parametrization 

,:v,^ n-"-. ,(BM) = { ( ) } , 

where (T, k) is the smallest positive solution of Pell's equation — Ak'^A = 4 (the minimality of 
(T, k) ensures that the image of (p consists of primitive conjugacy classes only). 

We are left to determine, for each primitive hyperbolic g £T satisfying (3.1) and having positive 
entries, the set oih ^ g with positive entries, having {g} = {/i}. Assume therefore that h = 7~^fi'7. 
Then 7 maps ah onto (as it can be seen by looking at what 7 does to the endpoints of a^, ag), 
and we therefore have that 7/5 G a^. Replacing 7 by g^^ for an appropriate n, we can assume that 
"fp G {p ^ gp). Writing g = 77', it follows that h = j'^. Since 7 maps ah onto a^, and since the 
point gp = 77V £ ilP ~^ 11' IP)-, it follows that 7'p £(/>—;■ hp). 

Therefore the number of hyperbolic h with {/i} = {5} and G I is the same as the number of 
points ^p £ {p ^ gp) (open geodesic segment) with 7 G F (compare with Lemma 1 of [5]), that is 
the same as the number of decompositions g = 77' with 7, 7' having positive entries (the positivity 
follows from 7/), 7'p € I and (10.1)). We will show that there are 0, 1, or 3 such points, depending 
on arithmetic conditions on A. 

Let X, Y, Z be the coordinates (2.1) for such a 7 G L, so X, Y, 2Z G N. We have 

2Z -X B Bo 
^^^"^ ^ 2Z^ = C = C'o' 
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therefore IZ = X + ni?o, 2Z = 1" + uCq. The equation \XY — = 3 becomes, after setting 

- 4n2A = 9. (A.5) 
In terms of solutions (t, u) with t > we find 

2Z = i(t + 2n(5o + Co)), X = ^(t + u(2C7o-5o)), F = ^(t + ^x(2So - Co)). (A.6) 

If (3, u) = 1, the sign of u is determined by the condition t = 2u{Bq + Co) (mod 3) which ensures 
that X, y, 2Z G Z. Notice that X, y > 0, so the triple (X, y, Z) indeed determines a matrix 7 with 
7P G Og. By (2.2) we find 

cosh(i(p,7/)) = |. (A. 7) 

We distinguish two types of solutions, depending on whether 3 divides u or not. 
Case I: 3 divides u. Letting u = 2>u\ t = 3t' , with {t',u') a solution of 

t'2_4An'2 = l, 

we have from (A.6) that 2Z > X, 2Z > Y when u' > 0, so the point 7p is on the same side of as 
gp. Since d{p, 7p) = t' , to determine when 7p € {p ^ 9P) we distinguish two cases : if the minimal 
positive solution {T,k) of (A. 2) has k even, then 2t' = T,2u' = k and d{p,gp) = 2d{p,jp), and we 
find that jp is the midpoint of {p — )• gp). On the other hand if k is odd, then the minimal solution 
{t', u') has t' = ^ — 1, in which case 'j = g and there are no points 7p on {p — t- gp). 

Case II: (3, u) = 1. In this case we necessarily have (3, A) = 1. Writing (A.5) as N(a) = 9 with 
a = t + 2m\/A, and assuming a solution oq = to + 2uo^/A exists with (3, uo) = 1, then all solutions 
are a = ao ("5" + k^/A) , with (T, k) a solution of (A. 2) with k even. The corresponding points 7/3 to 
these solutions lie on the axis Og at distance ^d{p,gp) apart if the minimal solution (T, k) of (A. 2) 
has k even, or at distance d{p, gp) apart if the minimal solution has k odd. In the former case we 
find two points jp on {p gp), at distance ^d{p,gp) apart, while in the latter only one. These are 
distinct from the point found in Case I, since here cosh d{p,jp) is not integral by (A. 7). 

We conclude that there is a partition of the set Dp in (A. 4): 

where Dp''^, respectively Dp''' is the subset of A for which the minimal positive solution {T,k) of 
(A. 3) has k even, respectively odd, and Dp^, respectively Dp^, is the subset of A such that (A.5) 
has a solution with (3, n) = 1, respectively it does not have such a solution. From the preceding 
discussion we conclude that the restriction of the parametrization 

is 1-1 if A € Dp'^ (no lattice points on {p — gp)), 2-1 if A G Dp'^ or A G Dp'^ (one point on 
{p — )• gp)), and 4-1 if A G Dp (three points on {p — )• gp)). We denoted by TZ^^"^ the image of if 
restricted to T>/\. 

Examples. I. A = 3, Pa = {(2, 1), (1, 2)}. The minimal solution of - I2k'^ = 4 is (T, k) = 
(4, 1) with k odd and 3|A so A G -Dp'^. Therefore there are two conjugacy classes in Ti^^™ with 
representatives: ip{2, 1) = ( } 3 ) , 93(1, 2) = (| } ) . 

II. A = 7, Pa = {(3,1), (3, 2), (1,3), (2, 3)}. The minimal solution of - 28A;2 = 4 is {T,k) = 
(16,3), and the equation — 28u'^ = 9 has solution (t,u) = (11, ±2), so A G Dp'^. From Case II 
we find 

g = ip{3,l) = 77', h = ip{3,2) = 7'7, for 7 = ( } i ) , 7' = ( } |) , 
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SO there are two conjugacy classes in 7^^^'™, {g} = {h} and {g} = {h}. 

III. A = 21, Va = {(5,1), (5, 4), (1,5), (4,5)}. The minimal solution of - 84^^ = 4 is {T,k) = 
(2 • 55, 12) and 3|A, so A G L»p'\ From Case I we find 

5 = <^(5,1)=77', h = ^{3,2)=j'j, for7 = (i|),y = (li), 
so there are two conjugacy classes in TZ'^^"^, {g} = {h} and {g} = {h}. 

IV. A = 13, Va = {(4,1), (4, 3), (1,4), (3, 4)}. The minimal solution of - 52A;2 = 4 is {T,k) = 
(2 • 649, 180), and the equation - 52u^ = 9 has solution {t, u) = (29, ±4), so A G Dp'°. We have 

¥^(4, 1) = ( ITo ) = 7171 = 7272 = 7373, 

with 71 = (ii),72 = (i|),7^ = (iifi),73 = (If i'i)),7^ = Hp. The first decomposition 
comes from Case I, while the last two from Case II. We have ip{l, 4) = 7171, 9?(3, 4) = 7272, ^{4:, 3) = 
7373, so 7^^3™ contains one conjugacy class. 

In Case I, the following lemma determines directly the midpoint of (p — )■ gp). 

Lemma 18. Let 9 = {c d) ^ hyperbolic matrix whose axis passes through p, having positive 
entries. Then a matrix 7 € SL2(M) such that -yp is the midpoint of {p — t- gp) has coordinates 
{X,Y,Z) given by (2.1) as follows 

X = A + \B, Y = D + \C, Z = \A + B = \D + C. 

Appendix B. Asymptotic behavior of the pair correlation density function 

In this appendix we give partial results towards proving the asymptotic formula (1.4), using the 
standard method of decomposing an automorphic kernel in terms of the spectrum of the hyperbolic 
Laplacian [14, Ch.l2]. Let P be a Fuchsian group of the first kind with fundamental domain having 
area Vp; and fix a point w G H. 

Recall the function /x(^) defined in (1.2), namely fx is the unique continuous function on [0, 00) 
with /x(0) = 0, which is smooth except at the two points ri < r2 such that sinh(ri) = X and 
2sinh = X, and whose derivative is 

if r < ri, 

if ri < r < r2, (B_X) 
if r > r2. 

Let u{z, w) = 4 im"zTm w ' ^° "^^^^ = 1 + 2u{z, vu), and define a point-pair invariant kernel 

function kx for fixed X > by 

kxiu{z,w)) = fx{d{z,w)). 

Let Kx{z,w) = Yli'yi^r^xiuiz^^w)) be the corresponding automorphic kernel. Conjecture 1 
states that the pair correlation density function for angles of the hyperbolic lattice Tlo is given by 

^^(^)=^^^^"''")' 

and we have to show that 

i^x(w,w) ~ -^X^, asX^oo. (B.2) 



fx{r) 



2 sinh(r) 



A/sinh2(r)-X2 



sinh(r) 
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Figure 3. The kernel function kxiu) for X = 2. 

Prom now on we fix X and drop the subscript X from the notation. Let h{t) be the Selberg/Harish- 
Chandra transform of the kernel function k{u), so that we have (see the proof of Theorem 1.16 in 
[14]) 

h{t) = / k{u{i,z))y'dfi{z), 
Ja 

where s = ^+it and dfi{z) is the hyperbolic measure. Using hyperbolic polar coordinates r = d{i, z) 
and Lp, we have y = (coshr + sinhr cos2(^)^^ and 



h{t) 



^ /■°°^ 2sinhr 

Jo (coshr + sinhr cos 293)' 



drdif. 



Let F(a, h; c; z) = J2n 



{a)n(b)n 



S^O n!(c). 



z"" be the Gauss hypergeometric function. We have: 



/■27r 

/ (coshr + sinhr cos ipY'd^ = 27re"F (s, - e^^) = 27re-"F (s, - e'^^) , 
so that 

h{t) = 27r / fx{r) sinh(r)e^'"'F (s, i; 1; 1 - e'^^) dr. (B.3) 
By definition the function h is even, which can also be seen from (B.3) via the relation 
Fis i- 1- 1 - = ^^^/'^~^^ F(s i- s + i- + IfcllZ^x^/^-s p(i_g 1 . 3 _ . ^ 

-c 2' r(i/2)r(i - s)^ 2' * ^ 2' -^y ^ p^2/2)r(s) ^ *'2'2 

for < X < 1, s / i. 

We have the following estimates for h. Notice that to obtain absolute convergence in the 
spectral decomposition of the kernel Kx{z,w) and finish the proof of (B.2), we would need 

h{t) < (l + |^|)-2-^ 

Lemma 19. (a) h{^) = nX^ and for s = \ + ite (0, 1) we have h{t) < x-^^^^^-s^+s)^ 

(b) There exists e > such that h{t) is holomorphic in the strip \ lmt\ ^ 5 + e '^'^'^ satisfies 
Kt) <- for t G M. 

Proof, (a) For t = | we have s = and /i(§) = 27r JJ^ /x(r) sinh(r)(ir. Since fx is continuous, 
integrating by parts we obtain 



Ml) 



-27r / /^(r) cosh(r)(ir. 
/o 
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We break the integral in three parts, at the two points ri < r2 where fx is not smooth. Using (B.l), 
we find that the three integrals are convergent and can be explicitly computed by the fundamental 



theorem of calculus, yielding = 27r y sinh^(r2) — sinh^(ri) = vrX^. 

For s = I + it G (0, 1), we use formula (B.3) and break the integral in two parts at the point ri 
as in the previous paragraph. For s / | we have ^1, 1, l) = r(i/2)r(i-s) ' '^^il^ ^, 1, x) <C 
— ln(l — x) as X —7- 1, obtaining 

/ fx{r) sinh(r)e-'^'F (s, i; 1; 1 - e'^'') dr <^XlnX. 
Jo 

if s 7^ ^, with an extra factor of ln(X) if s = ^. 

For r > ri, the integrand in (B.3) is bounded above by sinh(r)X^'^, and computing the 

integral as in the case 5 = yields the claim. 

(b) From the identity preceding the lemma, we have h{t) = hi{t) + hi{—t) where (s = ^ + it) 

hi{t) = r(i/2)r(i~-1) r ^""^"^ ^^^Hr)e-''F {s, i; . + i; e'^'^) dr. 

Using the power series expansion of sinh(r)e~^'^F (s, ^; s + ^; e"^*") , integration by parts gives 

^r(l/2-.) {sUl/2)n ^ e-(^»+-i) e-(^"+-+i) ^^ 

''^^*^-r(l/2)r(l-s)yo \^^n!(. + l/2)„ 1^2n + .-l 2n + s + 1 J * 

Denote by Gs{r) the series inside the integral. We proceed as in part (a), breaking the integral at 
the points ri = ri{X) and r2 = r2{X). From the formula (B.l) for f'^{r), we easily conclude that 

f'^{r)Gs{r)dr<.^^, / /^(r)G,(r)dr « 



by integrating by parts, using the series expansion of Gs{r). In the middle range we have 

fxir)Gs{r)dr <^ 



3/2 ■ 



using that sinh(r) e^'^^dr <^ -Xn- Since ^p!(^ f-* <C Isl ^Z"^, the estimate in part (b) 

1 y sinh^(r)-smh^(ri) *' J- U '^i 

follows. □ 
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